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Our  overall  goal  Is  to  define  computational  techniques  to  be  used  by 
a system  In  making  a visual  scan  of  a dynamic  environment  with  which  It 
Is  to  Interact.  Here,  ye  discuss  both  brain  mechanisms  In  the  visual  t 

y ^ * 

systems  of  animals  and  humans  and  computer  techniques  for  the  analysis  J 
of  color  photographs  of  natural  scenes.  We  present' schemas  as  a formaliza- 
tion of  the  system's  'knowledge  units'.  This  notion  Is  helpful  for  ^ 

our  work  In  both  the  BT  (Brain  Theory)  and  AI  (Artificial  Intelligence) 

c <* 

approaches . We  further  present  specific  studies — from  our^own  group 

and  from  elsewhere — of  subsystems  of  both  animal  and  computer  visual 

( 

systems.  -We  shall  examine _the  Interaction  of  high-level  processes 
with  low-level  systems,  as  part  of  a general  emphasis  on  Integrated 
system  design.  Part  II  (Rlseman  and  Arblb  (19761)  will  focus  on 
techniques  for  segmenting  single  static  colored  Images. 


This  work  was  supported  In  part  b^^JTIH  grant  SRM  N^09755-06  COM 
and  by  ONR  grant  N00014-75-C-0459  and  NSF  grant  DCR75-16098. 
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>roc»9«e8  to  construct  a nodal  of  the  three-dimensional  world  represented 


iMge  Specific  Model 


Semantic  Hate  Base 


•rc  duplicated  acroaa  the  entire  array.  Functions  operate  on  the  256  This  general  transformation  of  sensory  data  Into  different  representa- 

tions of  increasing  abstraction  is  similar  to  the  structure  found  in 

grid  of  image  data  and  reduce  it,  layer  by  layer,  to  single  cells  each  HEARSAY,  a computer  speech-understanding  system  that  we  discuss  further 

In  Section  4. 

of  which  contains  information  extracted  from  the  entire  scene.  Many 


p«rBsn«ncly  •tored  in  the  scmentic  date  base  (*long  term*  storage)  and 
laage'Speclflc  data  (short  tens  storage)  in  the  fona  of  the  RSE  data. 

The  aodel  vlll  contain  hypotheses  about  surfaces  (and  volumes),  as  well 
aa  schemas  for  objects  and  appropriate  frames  (representations  of  familiar 


I 


of  ch«  original  pattern  some  small  distance  (without  changing  the  pattern 


Figure  5:  Segmentation  on  Disparity  Cues. 

(a)  The  two  upper  rows  yield  the  five  rows  below;  with  a 
1 occurring  every  time  the  corresponding  'retinal  inputs' 
match  with  the  disparity  that  labels  the  row.  The  Job  of 
the  segmentation  routine  Is  to  suppress  spurious  regions  like 

C and  E,  so  that  B,  A and  D are  available  to  high-level  systems 

(b) -(d)  Actual  simulation  of  our  model  for  segmentation  on 
disparity  cues.  The  right-hand  side  shows  the  input  read  in 
by  touching  a light  pen  to  the  disparity  lines  (compare  the 
displacement  terms  in  (a)).  The  system  then  computes  the 
correlations  for  different  disparity  shifts — as  shown  in  the 
5 left-hand  displays.  Excitatory  interaction  within  each 
array  and  interaction  with  the  inhibitory  array  dramatically 
brings  the  signal  out  of  the  noise  as  we  go  from  (b)  through 

(c)  to  (d). 
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difference  in  spatial  location; k shows  the  Inhibitory  interaction 
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cone'  acLlvity*  vlth  resultant  diminution 


while  merges  with  disjoint  sets  of  labels  are  never  allowed.  Otherwise 


c*C«d  on  tb*  ability  to  daflna  ncanlngful  conatralnla(  relationahlpa  that  the  problea  has  proven  intractable  to  ao  aany  previous  atteapts*  oi 

bntweao  objects.  Many  of  those  used  by  Tenenbaua  and  Barrow  arc  basically  should  consider  the  ways  to  relax  the  deaands  for  complete  segmentation 

2D  special  relationships,  and  since  this  simple  scene  Involved  only  the  prior  to  Initial  labelling  of  regions;  these  processes  might  Interact 

well  plans  (i.e.»  2D),  the  relstlonehips  were  sufficient  to  achieve  the 


Imsins  novelties.  If  the  shadow  erees  in  a road  scene  at  night 


p h««  been  replaced  by  an  aaeignmenc  to  each  point  of  feature  descrlptora 


visual  Input  •*  Spatial  Maps  of  Light  Intensity 
Possibly  Varying  Over  Tisc 
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